Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark for C++ PDQ index #1699

Merged
merged 19 commits into from
Dec 18, 2024
Merged

Benchmark for C++ PDQ index #1699

merged 19 commits into from
Dec 18, 2024

Conversation

16BitNarwhal
Copy link
Contributor

@16BitNarwhal 16BitNarwhal commented Nov 28, 2024 •
edited
Loading

Summary

Resolve #1697

C++ query benchmark program for brute force linear scan and mih (pdq/cpp/index/mih.h)
Benchmark implementation located in pdq/cpp/bin/benchmark-query.cpp
Benchmark results in pdq/cpp/index/README.md

To run:

cd pdq/cpp
make benchmark-query
./benchmark-query

Sample results (ran on Ubuntu 24.04.1 LTS, Intel Core i7-14700KF with 20 cores, 28 threads, 64GB RAM):

$ ./benchmark-query -m linear
METHOD: Linear query
QUERY COUNT:             1000
INDEX COUNT:             10000
TOTAL MATCH COUNT:       1000
TOTAL QUERY SECONDS:     0.258911
SECONDS PER QUERY:       0.000259

Dcallies
Dcallies requested changes Dec 2, 2024
View reviewed changes
Copy link
Contributor

@Dcallies Dcallies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only some minor inline comments that I think will block extension. Otherwise looks good, thanks for doing it and for doing a README!

Dcallies
Dcallies approved these changes Dec 18, 2024
View reviewed changes
Copy link
Contributor

@Dcallies Dcallies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my blocking comments resolved, let's shipit

pdq/cpp/common/pdqutils.cpp
Comment on lines +47 to +52
for (int i = 0; i < numBitsToFlip; i++) {
int bitIndex = bitIndices[i];
int wordIndex = bitIndex / 16;
int position = bitIndex % 16;
noisy.w[wordIndex] ^= (1 << position);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignorable: This looks uniformly random to me!

@Dcallies Dcallies merged commit eefe31a into facebook:main Dec 18, 2024
4 checks passed
Mackay-Fisher pushed a commit to Mackay-Fisher/ThreatExchange that referenced this pull request Dec 18, 2024
@16BitNarwhal @Mackay-Fisher
[pdq] Benchmark for C++ PDQ index (facebook#1699)
f668729
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ianwal ianwal ianwal left review comments

@Dcallies Dcallies Dcallies approved these changes

Assignees
No one assigned
Labels
CLA Signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[pdq][mlh] Add querying benchmark to /bin
4 participants
@16BitNarwhal @Dcallies @ianwal @facebook-github-bot